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PREFACE 

This  paper  reports  the  partial  results  of  a survey  conducted  during  the 
spring  and  summer  of  1977  on  how  the  U.S.  aerospace  Industry  manages  Its 
software  engineering  projects.  The  survey  was  conducted  on  a sample  set 
of  the  U.S.  aerospace  Industry  as  represented  by  membership  In  the  American 
Institute  of  Aeronautics  and  Astronautics  (AIAA)  Technical  Committee  on 
Computer  Systems.  The  Impetus  behind  this  survey  was  to  collect  data  for 
analysis  In  the  preparation  of  paper  on  software  engineering  project 
management  to  be  presented  at  the  AIAA  Conference,  Computers  In  Aerospace, 
31  October  through  2 November  1977,  In  Los  Angeles,  California.  This 
report,  which  can  also  be  found  in  the  conference  proceedings,  A Collection 
of  Technical  Papers,  under  the  title  "Software  Engineering  Project 
Management:  A State-of-the-Art  Report"  represents  only  a portion  of  the 
data. 

Time  constraints  dictated  that  the  paper  be  at  the  printers  by  23 
September  1977.  On  6 September  1977  decision  was  made  "to  go  with"  the 
data  on  hand,  which  represented  29  surveys  and  18  companies.  By  the  time 
the  actual  presentation  was  given  on  1 November  1977,  fifty-two  projects 
were  reported,  representing  33  companies,  or  70%  returned. 

It  is  anticipated  this  report  will  be  rewritten  using  the  balance  of  the 
data  received  from  the  aerospace  corporations  that  participated  in  the 
survey.  Not  only  the  data  that  was  available  on  1 November,  which  was 
used  in  the  presentation  itself  at  the  conference,  but  other  data  that  is 
still  coming  in  as  of  this  date. 

The  title  of  this  report  was  changed  on  the  cover  sheet  to  more  accurately 
reflect  the  contents,  and  to  draw  attention  to  the  fact  that  this  is  an 
interim  report.  It  is  also  anticipated  that  because  of  the  voluminous 
amount  of  the  data  received  in  the  survey,  other  reports  will  be  made,  not 
only  by  the  original  authors,  but  by  other  members  of  the  AIAA  Technical 
Committee  on  Computer  Systems. 


Richard  H.  Thayer,  USAF 
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Abstract 

Project  management  is  clearly  a part  of 
software  engineering,  and  its  effective 
employment  plays  a major  role  in  reducing 
the  problems  associated  with  delivering 
software  Cresponsive  to  the  users  needs) 
within  estimated  time  and  cost.  Software 
engineering  project  management  must  provide 
the  necessary  planning,  organization,  staff 
ing,  direction,  and  control.  The  question 
this  paper  addresses  is:  "What  is  the 
state-of-the-art  in  software  engineering 
project  management  today?".  In  an  attempt 
to  provide  this  answer,  a survey  of  major 
U.S.  aerospace  corporations  was  made.  In 
response,  highly  qualified  individuals  re- 
ported on  how  they,  their  company,  or  de- 
partment managed  large  software  engineering 
projects.  This  paper  includes  survey  re- 
sults describing  how  these  corporations 
manage  software  development  projects,  dis- 
cussion and  comments  on  the  major  differ- 
ences in  the  methods  used,  and  opinions  on 
how  software  engineering  project  manage- 
ment might  be  improved. 

Introduction 
Software  Engineering. 

The  failure  of  many  software  development 
projects  to  be  delivered  within  schedule 
and  cost,  responsive  to  the  users  needs 
and  to  exhibit  a high  degree  of  maintain- 
ability and  reliability  is  well  documented 
and  a major  topic  of  discussion  when  any 
group  of  programmers  gets  together. 

Software  engineering  was  introduced  in 
the  1970's  to  apply  the  stronger  disci- 
plines of  engineering (in  contrast  to  the 
"Art  of  Computer  Programming")  to  the  de- 
sign of  computer  software  in  an  attempt  to 
solve  the  problem  of,  or  at  the  very  least 
reduce  the  severity  of,  software  develop- 
ment failures.  As  a result  of  this  ap- 
proach, improvements  have  been  made  in 
software  development  techniques  which  have 
directly  aided  the  programmer. 

Similar  advances  have  not  been  heralded 
in  the  area  of  software  engineering  pro- 
ject management,  and  only  a few  new  tech- 
niques unique  to  this  task  have  been 
developed. 

Increased  emphasis  on  the  management  as- 
pects, as  far  as  the  authors  can  determine, 
dates  from  the  NATO  Conference  on  Software 
Engineering  in  1968,  and  its  successor,  the 
1969  NATO  Conference  on  Software  Engineer- 
ing Techniques  from  which  the  following 
quotation  was  obtained. 
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In  almost  no  department  of 
computer  science  was  it  required 
that  every  student  take  a course 
in  operations  research  and  manage- 
ment science  and  yet  I claim  that 
to  be  a fundamental  course  in 
software  engineering;  as  funda- 
mental as  automata'  theory  and 
more  fundamental  than  any  mathe- 
matics course  I can  think  of. 

[Perlis,  Report  NATO  Conference 
on  Software  Engineering  Techniques. 
TWl 

Since  that  time  the  procedures  generally 
grouped  under  the  heading  of  "modern  pro- 
grammer productivity  -techniques"  have  had 
a wide,  though  not  necessarily  even,  imple- 
mentation by  software  development  activi- 
ties. Have  management  techniques  changed 
to  keep  pace,  or  do  the  old  procedures 
still  apply?  And,  just  what  is  the  trend 
and  the  forms  in  software  engineering  pro- 
ject management  today?  These  are  the  maj- 
or questions  we  have  attempted  to  address. 

Data  for  this  paper  was  collected  from 
the  members  of  the  AIAA  Technical  Commit- 
tee on  Computer  Systems  who  volunteered  to 
participate  in  a project  management  sur- 
vey. Our  purpose  is  to  provide  a state-of- 
the-art  report  on  how  software  engineering 
projects  are  managed  in  the  firms  of  the 
aerospace  industry  in  today's  environment. 
The  paper  includes:  1)  survey  results  in 
condensed  form,  2)  a discussion  and  com- 
ments on  the  major  differences  in  the 
methods  used,  and,  3)  an  opinion  on  how 
software  engineering  project  management 
might  be  improved. 

Approach 

The  approach  taken  to  gather  data  was 
to  undertake  a descriptive  survey.  A de- 
termination was  made  that  the  members  of 
the  American  Institute  of  Aeronautics  and 
Astronautics  (AIAA)  Technical  Committee  on 
Computer  Systems  would  provide  an  excel- 
lent source  and  sample.  Committee  members 
represented  47  major  corporations,  or  maj- 
or corporate  sub-divisions,  and  occupied 
top  positions  (e.g..  Manager,  Software  and 
Computer  Systems  Engineering;  Manager,  Ad- 
vanced Computer  Technologies;  Manager, 
Scientific  Programming  Department)  within 
the  firms.  There  can  be  no  doubt  that  the 
participants  were  in  an  ideal  position  to 
report  how  the  U.S.  aerospace  industry  man- 
aged its  software  development  projects. 

Theoretical  Model  of  a Software  Engineering 
Project 

It  was  the  intent  of  the  authors  to 
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identify  and  probe  every  aspect  of  software 
engineering  project  management.  After 
rather  lengthy  biographical  research  it  was 
concluded  that  no  single  reference,  or  group 
of  references,  was  broad  enough  to  cover  the 
entire  subject.  Therefore,  to  assist  in  de- 
signing a questionnaire  that  was  "complete" 
the  authors  built  a theoretical  model  util- 
izing management  processes,  non-management 
processes,  activities  and  products  as  ele- 
ments. This  model  served  as  the  basis 
from  which  a full  range  of  questions,  with 
their  associated  possible  responses,  was 
developed. 

The  Survey 

The  survey  was  conducted  through  a 
rather  lengthy  written  questionnaire  con- 
taining 229  numbered  questions.  But,  be- 
yond that,  by  using  "packing  techniques" 
approximately  925  separate  responses  were 
possible.  The  survey,  running  to  72  pages, 
was  divided  into  three  parts.  Part  One 
dealt  with  defining  the  total  organization, 
management  structure,  requirements,  and 
philosophy  of  the  firm.  It  was  intended  to 
be  answered  by  top  management  and  provide 
the  backdrop  against  which  the  individual 
projects  would  be  viewed. 

Part  Two  concerned  questions  about  indi- 
vidual projects  aimed  at,  and  Intended  to  be 
completed  by,  the  project  manager.  It  was 
also  assumed  that  the  project  being  report- 
ed on  was  or  was  almost  completed.  This 
part,  containing  the  bulk  of  the  questions, 
was  divided  into  8 sections.  Segmentation 
was  accomplished  by  grouping  the  questions 
under  the  functional  headings  of  planning, 
organizing,  staffing,  directing  and  control- 
ing,  and  the  technical  categories  of  project 
identification,  requirements  specification 
and  project  results. 

Part  Three,  consisting  of  general  ques- 
tions not  project  specific,  but  calling  for 
evaluations,  opinions  and  suggestions,  was 
also  intended  to  be  completed  by  a project 
manager. 

The  questionnaire  was  designed  to  make 
analysis  as  easy  as  possible  so  responses 
were  primarily  of  the  multiple  choice  or 
fixed  alternative  types  rather  than  open- 
ended  or  narrative  in  form.  However,  where 
the  authors  felt  the  possible  number  of 
alternatives  was  very  large,  or  judgements 
were  called  for,  open-ended  or  narrative 
type  questions  were  used. 

Method  of  Conducting  Survey. 

The  survey  was  designed,  written,  tested, 
and  implemented  in  the  spring  and  summer  of 
1977.  Initial  contact  was  made  in  May  of 
1977  to  determine  which  members  of  the  com- 
mittee would  be  interested,  willing,  and/or 
able  to  participate.  Forty-five  members, 
representing  35  companies,  agreed  to  re- 
spond. An  initial  draft  was  completed  in 
June  1977,  and  critiqued  by  approximately 
75i  of  the  total  committee  membership.  The 
returns  from  this  critique,  along  with 


other  corrections,  were  incorporated  in  the 
final  survey  completed  by  the  members  in 
early  September  1977. 

Accuracy  of  the  Survey. 

Of  the  three  basic  methods  in  which  this 
data  could  have  been  derived,  (observation, 
document/record  review,  and  survey)  the 
survey  afforded  the  only  practical  ap- 
proach. Though  observation  and  document/ 
review  would  have  been  more  accurate  and 
perhaps  provided  greater  uniformity,  the 
financial  and  time  constraints  alone  ren- 
dered this  approach  impractical.  The  sur- 
vey, our  only  possible  alternative,  carried 
with  it  several  possible  infirmities.  Good 
questionnaires  are  difficult  to  construct; 
it  has  been  estimated  that  it  takes  eight 
hours  to  write  one  good  question  with  its 
accompanying  selection  of  responses.  And, 
in  spite  of  the  authors  best  intentions, 
questions  will  not  always  mean  the  same 
thing  to  all  people.  Surveys  are  also  dif- 
ficult to  administer.  "Administer"  is  a 
euphemism  for  "get  back  on  time". 

Our  questionnaire  was  probably  no  better 
or  worse  than  other  surveys  in  this  re- 
spect. It  was  developed  by  two  individuals 
over  a period  of  three  months  and  reviewed 
by  perhaps  a dozen  experts  in  the  data  pro- 
cessing field  prior  to  publication.  Even 
before  the  returns  started  coming  in,  some 
problems  of  ambiguity  were  identified;  how- 
ever, based  on  the  sheer  magnitude  of  the 
survey  and  the  ability  to  compare  answers 
within  each  return,  most  of  these  ambigu- 
ities could  be  resolved. 

Often,  when  respondents  chose  "other" 
and  answered  a question  in  a narrative  form 
it  was  possible  to  place  the  response  in  a 
predefined  category,  or  establish  a new 
category  in  which  other  "other"  responses 
could  also  be  accumulated. 

In  Instances  where  the  answer  to  a pre- 
ceding question  dictated  the  response  to 
following  questions  that  were  left  blank, 
the  answer  was  provided  to  assure  tracking. 
An  example  of  this  might  be  a series  of 
questions  concerning  HIPO  diagrams  in  which 
only  the  first  question  was  responded  to 
with  "Did  not  use  HIPO  diagrams". 

The  above  procedures  were  applied  in  the 
absolute  minimum  number  of  cases,  and  we 
are  certain  that  they  did  not  distort  the 
respondents  intentions.  In  some  few  in- 
stances where  uncertainty  still  existed,  we 
either  eliminated  the  response  or  confirmed 
the  interpretation  over  the  phone. 

We  believe  that,  though  some  inaccur- 
acies exist  due  to  interpretation,  in  ag- 
gregate the  study  meets  .its  objective  and 
does  reflect  the  state-of-the-art  in  the 
management  of  software  development  projects 
as  practiced  in  the  aerospace  industry  to- 
day. 
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Survey  Results 
Participation. 

In  keeping  with  the  highest  traditions 
of  software  development  there  were  both 
slippages  and  overruns.  Each  milestone  was 
slipped  by  one  week  and  no  amount  of  extra 
effort  or  midnight  oil  could  alter  this 
perverse  application  of  Parkinsons  Law  in 
which  work  expanded,  with  amazing  regular- 
ity, to  one  week  beyond  the  time  allotted 
for  its  completion. 

Of  the  4S  companies  surveyed,  slightly 
less  than  one-half  completed  the  forms  and 
returned  them  in  time  for  our  standard 
week- late  cut  off  date.  This  allowed  some- 
thing less  than  two  weeks  to  write  and  pre- 
pare the  final  paper  so  the  decision  was 
made  to  terminate  any  attempt  to  include 
data  from  late  returns  in  this  paper,  but 
rather  to  update  it  for  presentation  at  the 
conference  in  the  fall.  In  some  instances, 
where  we  have  yielded  to  temptation  and 
peeked  ahead,  we  have  found  that  no  signif- 
icant differences  exist  between  the  first 
29  projects  reported  on  and  the  subsequent 
arrivals . 

Since  each  firm  was  contacted  person- 
ally, both  before  and  during  survey  time, 
it  was  anticipated  that  a very  large  major- 
ity of  forms  would  ultimately  be  returned. 
It  was  not  a lack  of  interest  on  the  part 
of  the  participants,  but,  rather  the  short 
time  available,  the  complexity  of  the  ques- 
tionnaire, and  the  prevalence  of  late  sum- 
mer vacations  in  the  upper  reaches  of  in- 
dustry that  have  contrived  to  keep  the  ini- 
tial response  at  just  under  SOI. 

Each  participant  was  given  the  option 
of:  complete  anonymity  through  full  dis- 
semination of  their  returned  survey.  In 
virtually  every  instance  anonymity  was 
opted  for,  so  no  reference  to  specific 
firms  will  be  made  in  the  body  of  the  re- 
port and  only  summary/statistical  data  will 
be  maintained  or  made  available  to  inter- 
ested parties. 

The  Firm. 

Of  the  initial  returns,  approximately 
751  of  the  firms  were  engaged  primarily  in 
manufacturing  and  151  in  engineering  and 
technical  support  services.  There  was  also 
one  software  house  and  one  government  a- 
gency.  Average  income  for  the  firms  rang- 
ed from  "Between  One  and  Ten  Million  Dol- 
lars" (one  response)  to,  "In  Excess  of  One 
Billion  Dollars"  (two  responses).  A rough 
approximation  of  mean  income  for  all  the 
companies  included  in  this  report  is  290 
million  dollars  annually,  no  mean  sum.  On- 
ly a small  percentage  of  this  income  was 
derived  from  software  development,  however 
in  most  instances  being  less  than  101. 

From  the  standpoint  of  employees,  the 
number  ranged  from  a low  of  210  to  a high 
of  50,000,  while  employees  engaged  in  soft- 
ware development  activities  showed  a low  of 


20  and  a high  of  1200  individuals,  with  177 
being  the  average. 

Several  questions  address  contract 
types:  What  types  were  used?  What  types 
were  preferred?  In  analyzing  this  no  spe- 
cific "winner"  emerged,  and  it  is  recog- 
nized that  many  factors  go  into  the  choice. 
Some  contract  types  were  cleat ly  held  in 
lower  esteem  than  others.  In  general,  cost 
plus  types  were  most  preferred  when  the 
company  was  in  the  position  of  contractor 
and  firm  fixed  price  the  preference"  when 
the  roles  were  reversed. 

What  emerged  as  the  most  preferred  form 
of  contract  was  the  two-phased  instrument 
in  which  phase  one  analyzes  requirements, 
determines  feasibility,  and  estimates  cost. 
Phase  two  directs  development.  Only  one 
firm  discouraged  their  use,  but  only  when 
they  were  contracting  for  development. 
Presumably  it  was  felt  that  the  expertise 
in-house  was  equal  to  the  task  of  providing 
phase  one  analysis. 

Only  one  firm  payed  incentives  for  early 
or  on  time  project  completion;  and,  where- 
as only  one  firm  reported  utilizing  pro- 
cedures in  which  programmers  or  analysts 
bid  on  specific  tasks  within  development 
projects,  two  reported  great  to  moderate 
success  with  this  technique. 

It  was  found  that  in  aggregate,  941  of 
the  development  team  members  were  permanent 
employees,  with  temporary  hires  and  consul- 
tants accounting  about  equally  for  the  re- 
maining 61.  Though  impossible  to  come  up 
with  an  accurate  percentage  figure  or 
ratio,  the  responses  indicate  clearly  that 
the  straight  applications  analyst  or 
straight  computer  programmer  is  in  a de- 
cided minority  (probably  less  than  251) 
with  analysts/prbgrammers  being  the  rule. 

Only  one  organization,  the  smallest 
with  20  individuals  engaged  in  software 
development,  reported  not  using  any  form 
of  manual  periodic  progress  reporting. 
Weekly  status  reports  were  the  most  com- 
mon - 771,  project  status,  next  - 611,  and 
significant  change,  third,  - 461. 

Forty-six  percent  of  the  firms  also  re- 
ported using  some  form  of  automated  pro- 
ject management  system.  Surprisingly,  the 
second  and  third  firms  in  size  from  a de- 
velopment staff  viewpoint  (averaging  650 
people)  did  not  use  an  automated  project 
management  system,  and  only  slightly  over 
201  of  those  responding  used  system  soft- 
ware to  monitor  development.  As  far  as 
could  be  determined  from  a company  stand- 
point, productivity  indexes  were  in  total 
aisuse,  though  it  was  found  later  on  that 
project  managers  did  on  occasion  employ 
these  techniques.  Approximately  701  of 
the  firms  responding  indicated  that  they 
had  employed  one  or  more  of  the  procedures 
generally  placed  under  the  heading  of 
"modern  programmer  productivity  tech- 
niques", though  only  401  appear  to  have 
made  a substantial  commitment.  Of  the 
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seven  techniques  listed  (tesm  concept,  de- 
velopment support  library,  HIPOs,  pseudo 
code,  walk-thru's,  top-down  design,  and 
top-down  implementation),  the  employment  of 
HIPOs  has  proven  least  accepted  with  only 
two  firms  reporting  their  use,  one  at  the 
101  level  and  growing,  the  other  at  21  and 
in  a decline. 

Cost  per  line  of  code  is  a mandatory 
data  processing  survey  question,  so  of 
course  it  was  included.  Costs  varied  from 
a low  of  $S.OO  to  a high  of  $330.00  - with 
an  average  at  $78.78.  It  is,  of  course,  a 
meaningless  statistic  given  like  this,  but 
it  is  easy  to  remember. 

Project  Identification. 

In  collecting  the  data  on  specific  pro- 
jects we  felt  it  important  to  obtain  re- 
sponses from  those  individuals  most  famil- 
iar with  the  development  effort  - the  pro- 
ject manager  or  a senior  member  of  his 
staff.  In  keeping  with  our  requests  that 
these  "front  line"  supervisors  provide  the 
inputs,  781  of  the  respondents  were  indeed 
project  managers  with  the  remaining  221 
occupying  supervisory,  staff,  or  technical 
positions  just  one  step  removed. 

As  for  the  projects.  Table  1 provides  a 
summary  of  the  application/functional  areas 
into  which  they  fell.  Many  projects  cover- 
ed more  than  one  application/functional 
area  prompting  us  to  come  up  with  an  addi- 
tional category,  "Multi-functional  command 
and  control  systems."  In  this  category  we 
placed  those  projects  that  the  respondent 
indicated  covered  many  areas  centering 
around  a military  command  and  control  ap- 
plication. 


Table  1 

Appllcatlon/Funcclonal  Areas 


Appllcatlon/Functlonal  Area  Pet  Reported 


Commercial/business  7 
Data  acquisition/retrieval  3 
Scientlf Ic/data  reduction  14 
Process  control/embedded  computers  28 
Command  and  control  systems  21 
Management  Information  systems  3 
Communlcstlon  systems  3 
Computer  systems  software  3 

Multi-functional  command  and  17 
control  systems 


About  two- thirds  of  these  projects  in- 
volved totally,  new  software  development 


while  the  remaining  one-third  was  either 
the  continuation  of  a previously  completed 
software  system,  a major  modification  of  an 
existing  capability,  or  both.  About  54t  of 
the  projects  used  commercial  off-the-shelf 
computer  hardware,  42$  employed  special 
purpose  computers,  and  one  project  used  a 
combination  commercial/special  purpose 
system. 

As  would  be  expected  in  dealing  with  the 
U.S.  aerospace  industry,  78$  of  the  pro- 
jects were  contracted  for  by  the  Federal 
Government,  the  balance  being  for  other 
companies  or  in-house  use.  Table  2 lists 
the  contract  types  used  on  the  projects 
for  which  this  data  was  reported. 


Table  2 

Contract  Type  Used 


Contract  Type 

Pet  Used 

Firm  fixed  price 

18 

Cost 

12 

Coat  sharing 

4 

Coat  plus  Incentive  fee 

23 

Coat  plus  award  fee 

8 

Cost  plus  fixed  fee 

27 

Time  and  materials 

4 

Basic  ordering  agreement 

4 

These  contracts  involved  large  sums  of 
money  ranging  from  $200,000  to  half  a bil- 
lion dollars,  with  the  total  cost  of  all 
projects  surveyed  approximating  one  bil- 
lion dollars.  Software  development  ac- 
counted for  $301,000,000  of  this  amount, 
distributed  between  $30,000  and 
$159,000,000. 

With  few  exceptions,  all  of  these  pro- 
jects were  started  subsequent  to  1972, 
with  33$  being  completed  in  the  1975/1977 
time  frame.  Fifty-two  percent  of  the  pro- 
jects are  still  unfinished,  but  the  major- 
ity of  these  are  nearing  completion.  The 
languages  most  used  (by  a wide  margin) 
were  FORTRAN  and  some  assembly  language. 
Executable  lines  of  code  ranged  from  5,000 
to  1,200,000.  Total  executable  lines  of 
code  reported  on  all  projects  were 
4,073,800. 

Requirement  Specifications. 

Had  we  been  searching  for  someone  who 
had  been  provided  a perfect  set  of  re- 
quirement specifications  we  would  have 
come  away  sorely  disappointed.  As  a gen- 
eral rule,  the  larger  the  system,  the  more 


detailed  the  specifications.  Yet,  the 
more  detailed  the  specification  the 
greater  the  requirement  for  rewrite  prior 
to  proceeding  with  design.  We  asked  the 
question:  "On  a scale  of  1 to  7,  with  1 

being  little  more  than  the  name  of  the 
system,  and  7 being  complete  specifications 
down  through  preliminary  design,  how  de- 
tailed were  the  specifications  provided?" 

Combining  ratings  and  matching  them 
with  the  project  costs  and  specification 
rewrite  requirements,  we  came  up  with  the 
informntion  in  Table  3. 


Table  3 

Original  Specification  Detail  Vs  Percent  Rewrite 


level  of 
Specification 
Detail 

Average 

System 

Cost 

Pet  Spec  Rewrite 
Prior  to  Design 
Avg  High  Low 

Nr 

Resp 

2 4 3 

640,000 

21 

50 

0 

8 

4 4 5 

2,840,000 

27 

60 

0 

8 

6 4 7 

27,657,000 

35 

100 

0 

9 

Of  the  specifications  prepared  in-house 
by  the  project  managers'  own  organization, 
an  average  231  rewrite  was  required  where- 
as a 471  rewrite  was  called  for  when  the 
user  provided  the  specification.  The 
customer  or  customer  affiliate  prepared 
the  initial  specifications  in  about  301  of 
the  cases  reported,  the  organization  in 
601,  and  consultants  or  similar  third 
party  in  the  remaining  101. 

The  reasons  for  specification  rewrites 
were  about  equally  divided  between: 

"Original  requirements  ambiguous  and 
incomplete",  and,  "Continued  Government  re- 
direction and  problem  rescoping". 

One  manager,  however,  considered  speci- 
fications rewriting  an  immutable  rule,  a 
"normal  and  expected  iteration  of  total 
system  design".  Table  4 lists  the  methods 
and  techniques  reported  used  in  specifi- 
cation preparation. 

It  was  interesting  to  note  that  no 
apparent  relationship  existed  between  suc- 
cessful projects  and  who  originated  the 
specifications  or  the  degree  of  rewrite 
required. 

Documentation.  One  of  the  major  areas 
of  customer  concern  was  the  type  of  docu- 
mentation to  be  provided  as  part  of  the 
system.  There  was  no  discernible  relation- 
ship between  the  number  of  document  types 
required  and  system  size.  It  should  be 
noted  that  though  the  United  States  Govern- 
ment was  the  customer  in  the  majority  of 


Table  4 

Specification  Preparation  Techniquea 


Technique  Pet  Used 


Top-down  (Hierarchy  of  function)  32 

Structured  flow  4 

Formal  requirements  language  0 

Phases  where  design  and  coding  started 
before  specifications  were  complete  32 

In  a manner  to  facilitate  tracking 
software  development  from  requirements 
through  coding  44 

In  precise  measureable  terms  to  aid  In 
development  of  acceptance  tests  12 


cases,  the  number  of  document  types  speci- 
fied as  deliverable  end  items  did  not  dif- 
fer in  any  significant  way  between  Govern- 
mental or  non- Governmental  customers. 

Three  contracts  called  for  3 or  fewer  docu- 
ments while  all  the  rest  required  between 
5 and  13.  See  Table  5 for  a list  of  soft- 
ware documentation  required  by  surveyed 
projects  (28  projects  reporting). 


Table  5 

Software  Documentation  Required  by  Customers 


Document 

Pet  Required 

Object  listing 

75 

Source  listing 

93 

Functional  Description 

82 

Data  Requirements  Document 

50 

System/Subsystem  Specifications 

79 

Program  Specifications 

71 

Data  Base  Specifications 

50 

Users  Manual 

75 

Computer  Operations  Manual 

64 

Program  Maintenance  Manual 

32 

Test  Implementation  Plan 

75 

Test  Analysis  Report 

61 

Other 

29 

Specific  Customer  Requirements.  In  ad- 
dition  to  document  types  a host  of  other 
customer-defined  requirements  were  also 
specified.  These  included  standards,  tech- 


S 


niques,  limitations,  and  constraints,  and 
are  summarized  in  Table  6 (25  projects 
report ing) . 


Table  6 

Specific  Customer  Requirement 


Requirement  Pet  Required 


Specific  computer  AO 
Adherence  to  storage  limitations  36 
Adherence  to  speed  constraints  52 
Adherence  to  specific  language  A8 


Customer  participation  In  design  function  AA 
Customer  participation  In  coding  function  20 


Adherence  to  prioritized  requirements  23 
Development  under  llfe-cycle-costlng  8 
Development  under  deslgn-to-cost  8 
Development  under  programming  techniques  2A 
Portability  A 
Adherence  to  human  engineering  techniques  36 
Data  system  security  20 
MIL-STD  5279  (or  modified)  8 
Specified  types  of  review  68 
Specified  frequency  of  reviews  52 
Special  Input  data  AA 
Special  output  requirements  A8 
Test  plan/procedures  20 
A reliability  figure  12 
A maintainability  goal  0 
A warrantee  of  the  software  A 
A followup  maintenance  contract  2A 
Customer  training  32 


Planning. 

Of  the  total  time  available  to  the  pro- 
ject manager  and  his  staff,  approximately 
111  was  employed  in  planning  and  replanning 
In  most  cases  the  project  manager  was 
"brought  on  board"  shortly  after  project 
inception  and  participated  actively  in  the 
planning  activities.  Table  7 breaks  down 
the  total  planning  time  into  its  consti- 
tuent parts  (18  projects  reporting). 

In  191  of  the  projects  reported  the 
firm  provided  the  manager  with  a formal 
planning  guide.  In  431  of  the  projects  he 
war  assisted  in  planning  activities  by  the 
customer.  Though  two  small  projects  re- 
ported producing  no  planning  documents  at 
all,  the  range  and  number  of  plans  devel- 
oped by  the  remainder  is  impressive,  as 
can  be  seen  in  the  Table  8. 


Table  7 

Allocation  of  Planning  Time 


Planning  Function 

Pet  Time  Spent 

Developing  an  overall  project 


management  plan  22 
Developing  control  procedures  21 
Staff  planning  17 
Organizational  planning  16 
Quality  assurance  planning  12 
Administrative  planning  9 
Other  3 


In  planning  the  development  activities 
a number  of  recognized  tools  and  proced- 
ures were  employed.  In  two-thirds  of  the 
cases,  projects  were  divided  into  phases 
to  facilitate  planning,  and  62t  of  the 
projects  used  a work  breakdown  structure, 
generally  shredded  out  to  5 levels. 


Table  8 

Planning  Documents  Prepared 


Type  of  Plan 

Pet  Reported 

Software  development 

85 

Test 

78 

Training 

67 

Organization 

70 

Change  control 

70 

Staffing 

63 

Review  and  reporting 

59 

Resource  requirements 

50 

Documentation 

56 

Project  management 

A8 

Phase  and/or  delivery 

37 

Impl ementa  t Ion 

30 

Data  conversion 

A 

Other 

7 

None 

8 

Workload  charts  proved  to  be  the  most 
popular  tool  (used  on  57t  of  the  projects) 
followed  by  GANTT  charts  and  some  version 
of  PERT  (roughly  351  each). 
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Cost  estimation,  as  everyone  has  sus- 
pected for  some  time,  is  far  from  a sci- 
ence. Table  9 shows  the  methods  employed 
and  the  percentage  of  projects  on  which 
they  were  used.  Some  projects  combined 
methods.  For  those  clamoring  for  their 
own  crystal  ball  (fourth  method),  it  should 
ho  noted  that  the  projects  relying  on  that 
technique  were  each  6 months  late  and 
e.xperienced  significant  cost  overruns. 

There  is  every  indication  that  the 
development  of  quality  assurance  standards 
was  also  a major  activity  engaged  in  by 
project  personnel.  In  interpreting  Table 
10  (16  projects  reporting)  it  should  be 
noted  that  in  only  two  instances  were  both 
a company  standard  and  a project  unique 
standard  applied  at  the  same  time.  In 
every  other  case  if  the  company  had  a 
standard  it  was  used,  and  no  project  unique 
standard  was  developed  (or  maybe  the  other 
way  around) . 


Table  9 

Methods  of  Estimating  Cost/Schedules 


Method  Pet  Used 


Estimates  based  on  a similar  project  67 

Formula  A4 

Cost  and  schedule  dictated  15 

Crystal  b.all  (or  similar  means)  11 

Provided  by  someone  who  has  a knack 

for  estimating  correctly  4 

Simulation  techniques  0 

Other  11 


Only  205  of  the  participants  reported 
not  using  a quality  assurance  program 
while  the  remainder  divided  equally  be- 
tween formal  and  informal  programs. 

The  extent  to  which  "Modern  Programming 
Prat t ices" were  used  on  individual  projects 
exceeded  the  amount  reported  by  the  com- 
panies as  a whole.  This  probably  reflects 
our  request  that  projects  employing  these 
techniques  be  selected  for  the  survey. 

The  "newness"  of  these  practices  promoted 
us  to  include  a reference  in  the  table  to 
assure  a common  understanding.  Though 
every  project  reporting  in  this  area  employ- 
ed some  of  the  techniques  (two  small  pro- 
jects used  only  reviews)  we  did  not  really 
iisccrt.nin  if  a radical  shift  had  taken 
place  or  whether  many  respondents  were  do- 
ing substantially  what  they've  always  done, 
l)ut  under  a new  name.  HIPOs  are,  of 
course,  a major  departure,  but,  then  again, 
their  reporting  uses  were  not  very  great. 
Table  11  reflects  the  results  of  our 


Table  10 

Quality  Assurance  Standards 


Standard 

Pet  Company 
Wide 

Pet  Project 
Unique 

Documentation 

18 

75 

UBS  code 

44 

25 

Scheduling 

38 

31 

Performance  measurement 

13 

25 

Requirements  analysis 

13 

38 

Preliminary  design 

19 

56 

Detail  design 

25 

50 

Coding 

25 

50' 

Test  planning 

13 

56 

Software  verification 

13 

75 

Reviews  and  audits 

31 

44 

Configuration  management 

31 

25 

Discrepancy  and  correction 

19 

75 

Software  acceptance 

13 

56 

Table  11 

Modem  Programming  Practices 


Practice 

Pet  Reported 

Program  manager  authority  [Black,  1977]  78 

Reviews  [Black,  1977]  81 

Unit  development  folder  [Ingrassia,  1976)  37 

Design  discipline  and  verification 

[Black,  1977]  48 

Program  n>odularity  [Black,  1977)  70 

Naming  conventions  [Black,  1977]  56 

Structured  form  [Dljkstra,  1969]  IS 

Structured  walk-throughs  [Weinberg,  1971]  44 

Structured  analyslb)  [Yourdon,  1975]  15 

Structured  design  [Yourdon,  1975]  41 

Chief  programmer  teams  (Baker,  1972)  48 

HIPOs  (IBM,  1975]  11 

Support  library  and  facilities  44 

[Black,  1977) 

Phase  testing  [Black,  1977]  81 

Configuration  management  78 
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survey.  References  cited  contain  the 
definitions  for  tlie  purpose  of  this  paper. 

Tlie  software  development  tools/aids 
listed  in  Table  12  were  used. 


Software  Tools/Alda 


Pet  Used 


Software  Tools /Aids 


Structured  pre-compilers 
Automatic  flow  charters 
Library  monitors 
MACRO  programming  capabilities 
On-line  capabilities 
Automatic  test  case  generators 
No  software  tools/aids  employed 


The  test  tools/techniques/methods 
listed  in  Table  13  were  used  [for  defi- 
nition of  terms  see  Hartwick,  1977]  (23 
projects  reported]. 


Test  Tool/Techniques 


traditional  functional  approach.  A matrix 
organization  in  which  the  participants 
were  detailed  to  the  project  manager  for 
the  duration  of  the  task  accounted  for 
roughly  46t  of  the  cases  reported.  Just 
who  was  detailed  to  whom  usually  depended 
on  the  orientation  of  the  project  manager. 
If  the  manager  was  from  the  data  processing 
side  of  theheuse,  the  functional  experts 
were  detailed,  and  the  reverse  held  true 
(data  processing  experts  were  detailed]  if 
the  manager  was  assigned  to  the  functional 
activity.  In  two  cases,  however,  a distina 
entity  was  formed  and  both  functional  and 
data  processing  experts  were  detailed  to 
form  its  complement. 

A matrix  organization  in  which  work  is 
accomplished  through  tasking  line  or  staff 
agencies,  rather  than  having  individuals 
actually  detailed,  accounted  for  24t  of 
the  projects  reporting. 

An  often  voiced  concern  of  data  process- 
ing (DP]  managers  engaged  in  development 
activity  is  the  loss  of  control  that  may 
occur  when  programmers  and  analysts  are 
detailed  or  assigned  to  functional  manageis. 
In  311  of  the  projects  reported  the  DP 
manager  did  indeed  lose  control  of  DP 
people  to  some  unspecified  degree. 

Table  14  (with  29  projects  reporting] 
attempt  to  display  this  data.  In  con- 
structing the  array  we  separated  the 
straight  project  organization  from  its 
matrix  variant  and  subdivided  the  matrix 
form  into  its  two  basic  orientations,  task 
and  detail. 


Tools /Techniques 


Organization  Types 


Comparitors 

Editors 

Flow  charting 

I.ogl  c/cqunt  Ion  generators 

Program  structure  analyzers 

Correctness  proofs 

Symbolic  program  executions 

Initialization  tests 

Interaction  tests 

Arithmetic  tests 

Timing  analysis 

Branch  logic  tests 


Organization  Type  (By  Percent)  Matrix 

Line  of  Authority  Function  Pro]  Task  Petail 


Line  of  Organization 

Under  DP  manager  0 

Not  under  DP  manager  3 

Staff  Organization 

Under  DP  manager  0 

Not  under  DP  manager  0 


7 21 

11  18 


27  24  46 


Organ i za  t ion. 


There  were  a number  of  organizational 
variations  and  lines  of  authority  used  by 
the  survey  participants.  Some  form  of 
luoject  organization  was  the  overwhelming 
choice  with  only  one  firm  using  the  more 


In  every  case,  save  one,  the  development 
organization  was  subdivided  into  teams, 
each  under  the  direction  of  a technical 
leader  (average  4 teams  per  project). 

Though  analyst  programmer  teams  were  by 
far  the  most  common  form,  in  some  few 
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instances  teams  of  straight  analysts  or 
straight  programmers  were  also  used. 

Special  purpose  teams  were  also  em- 
ployed. Most  important  of  these  by  per- 
cent of  projects  were:  Test/product 
acceptance  - 34t,  integration  - 28t,  and 
interface  - 141. 

Where  test  teams,  or  similar  entities, 
were  used,  the  head  of  this  activity  usually 
reported  directly  to  the  project  manager. 
However,  in  S instances,  the  test  team 
reported  to  a more  senior  individual,  or 
to  some  other  outside  agency. 

Staffing 

Project  Manager.  Almost  invariably  the 
project  managers  were  selected  from  in- 
house  resources  with  approximately  one- 
lialf  coming  from  another  software  develop- 
ment project.  As  a general  rule,  appoint- 
ment to  the  top  project  position  was  made 
by  a senior  manager  not  in  the  DP  line  of 
authority.  The  typical  project  manager  had 
almost  11  years  experience  in  data  process- 
ing, though  the  range  reported  extended 
from  0 to  20  years.  The  years  of  experi- 
ence in  the  functional  area  of  the  project 
averaged  7.S  years  with  a range  of  0 to  16 
years.  Ninety  percent  of  the  project 
managers  professed  a working  knowledge  of 
FORTRAN  and  764  some  level  of  proficiency 
in  an  assembly  language.  Very  few  had  any 
knowledge  of  either  JOVIAL  or  COBOL. 

Tlie  average  age  of  project  managers  was 
38,  witli  a low  of  34  and  a high  of  50. 

Only  194  admitted  to  ever  having  been  a 
cliief  programmer.  As  Table  IS  attests, 
they  had  an  extremely  high  level  of  formal 
education.  Degrees  were  primarily  in 


Table  15 

Project  Manager  Education  Level 


Degree  or  Degree  Status  Pet 


BS/BA  degree  26 

Miisters  degree  44 

Masters  degree  plus  30  hours  (or 

doctoral  candidate)  15 

PHD  (or  equivalent)  15 


engineering  and  mathematics.  There  were  3 
physicists,  2 business  majors,  and  no 
degrees  in  computer  science. 

In  most  instances  it  was  necessary  to 
obtain  some  additional  training,  either 
prior  to,  or  early  in,  the  development 
cycle.  Table  16  provides  a breakdown  by 
subject  area  and  percent  involved. 


Table  16 

Project  Manager  Training 


Area 


Fct  Receiving  Tmg 


Functional  area  of  project 

59 

General  data  processing 

50 

H>dem  programming  techniques 

32 

Project  management 

59 

General  management 

59 

A specific  programming  language 

32 

Other  project  related  areas 

9 

The  Software  Development  Staff.  The 
source  of  programmers/analysts  was;  new 
hires  from  another  company  - 204,  new  hires 
from  school  - 74',  in-house  transfers  from 
another  project  - 454,  in-house  transfer 
from  non-project  oriented  activity  - 274. 
The  educational  level  of  the  programmers/ 
analysts  is  given  in  Table  17. 


Table  17 

ProgranuBers/AnalysCs  Education  Level 


Degree  or  Degree  Status 

?ct 

High  school 

< 1 

AA  degree  or  2 years  of  college 

11 

Between  2 and  4 years  of  college 

3 

BS/BA  degree 

56 

Masters  degree 

24 

Masters  degree  plus  30  hours  (or 
doctoral  candidate) 

2 

PHD  (or  equivalent) 

3 

Training.  In  784  of  the  projects  re- 
ported  the  project  manager  was  responsible 
for  identifying  training  requirements  for 
the  development  team.  By  far  the  most 
important  source  of  training  was  on-the-job 
training;  classes  conducted  by  team  members 
and  hardware/sof tware  vendors  came  in  a 
poor  second  and  third.  The  training  most 
often  required  was:  Programming  language - 
474,  and  operational  system  - 524. 

Of  8 projects  reporting  the  use  of  a 
programmer  support  librarian,  5 reported 
filling  the  position  with  a clerk/pro- 
grammer technician  while  the  remaining  3 
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used  programmers  in  this  capacity. 

The  personnel  turnover  reported  was: 
project  managers  - 64\,  functional/user 
analysts  - 431,  and  data  processing  ana- 
lysts - 141. 

Control 


The  process  of  controlling  a software 
engineering  project  may  well  be  the  most 
talked  about  and  least  understood  of  all 
the  project  managers'  functions.  Many 
projects  have  failed  because  of  the  mana- 
gers inability  to  adequately  define  pre- 
cisely where  the  project  was  in  the  pro- 
duction cycle.  Otlier  development  efforts 
have  been  cancelled,  not  for  inadequate 
programming,  nor  lack  of  technology,  but 
for  the  sheer  frustration  of  all  concerned 
in  attempting  to  determine  when,  if  ever, 
the  project  would  be  completed.  This 
portion  of  the  survey  attempted  to  deter- 
mine how  project  managers  control  and 
monitor  the  work  from  inception  to 
completion. 

Control  Techniques.  Early  in  the  sur- 
vey' we  asked  project  managers  what  type  of 
tools  they  used  in  planning  the  project. 

In  comparing  the  results  of  that  question 
with  the  analogous  query  on  control  it  was 
found  that,  by  and  large,  the  same  tools 
were  used  in  both  functions.  The  only 
significant  difference  was  the  decrease  in 
the  use  of  workloading  charts.  (See  Table  18) 


Table  18 

Systems  Used  In  Project  Control 


System 

Pet  Reported 

Milestone  tracking 

71 

Work  breakdown  structure  code 

70 

Uorkloadlng  charts 

36 

GANTT  charts 

32 

Modified  PERT 

29 

PERT 

4 

Other 

25 

No  systems  used 

21 

Reporting.  The  three  most  frequently 
used  manually  prepared  reports,  the 
originators  and  recipients,  and  the  percent 
of  projects  employing  them  were: 

Weekly  .Activity: 

ProTrammer/Analys*’  to  Project  Mgr  - 381 
Project  Mgr  to  Senior  Management  - 191 


Project  Status: 

Programmer/Analyst  to  Project  Mgr  - 191 
Project  Mgr  to  Senior  Management  - 431 


Significant  Change: 

Programmer/Analyst  to  Project  Mgr  - 51 

Project  Mgr  to  .‘Senior  Management  - 141 

A number  of  managers  reported  using  an 
automated  system  to  monitor  the  develop- 
ment effort.  In  541  of  the  projects  this 
system  was  used  to  accumulate  and  display 
data  such  as  manhours  by  activity.  An 
activity  was  defined  as  a part  of  a task; 
flow  diagramming,  coding,  etc.  Thirty-one 
percent  of  the  projects  reported  using  a 
task  oriented  system.  Beyond  that,  system 
software  was  used  to  check  individual  pro- 
grams, the  most  common  capabilities  being 
listed  in  Table  19. 


Table  19 

Automatic  Software  Monitoring  Capabilities 


System  Capability 


Pet  Reported 


Count  complies  per  module  S 
Count  lines  of  code  produced  15 
Check  adherence  to  coding  conventions  30 
Check  for  use  of  standard  data  names  25 


Other  projects  reported  using  productiv- 
ity indexes,  however,  these  were  manually 
derived.  The  most  frequently  used  indexes 
were:  line  of  complete  code  per  manhour, 
modules  completed,  and  program  errors. 

Table  20  lists  various  stages  into 
which  a software  development  project  may 
be  divided.  Our  question  asked;  "Which 
of  these  were  recognized  as  separate  and 
distinct  phases?",  xhe  percentages  indicate 
that  dividing  projects  into  phases  is  one 
of  the  major  methods  the  manager  employs 
in  "getting  a handle"  on  the  total  effort. 


Work  Assignment.  Tables  21  and  22  oro- 
vide  an"'6nffap5lll  1 Jed  report  on  how  work 
assignments  from  pioject  manager  to  team 
chief  to  individual  worker  are  generally 
handled.  Table  21  indicates  how  task 
assignments  were  made  to  teams  and  individ- 
uals, whether  completion  dates  were  speci- 
fied, and  whether  the  task  was  defined  in 
the  context  of  the  project.  The  questions 
that  provided  the  framework  for  Table  22 
attempted  to  ascertain  the  degree  of  in- 
teraction or  feedback  permitted  in  estab- 
lishing required  or  estimated  delivery 
dates . 

In  631  of  the  projects  reporting,  tasks 
were  assigned  to  teams  and  individuals  as 
soon  as  they  were  sufficiently  developed 
and  defined.  Twenty-four  percent  reported 
making  work  assignments  as  resources  became 
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available  and  the  remaining  project  man- 
agers made  task  assignments  on  a regular 
recurring  basis,  e.g.,  weekly,  bimonthly, 
etc. 


Table  20 


Software  Development  Phaaes 


Pet  Raportad 


Pet  Reported  Time  and  task  aaatgned 


System  definition 

56 

Requirements  definition 

81 

System  design 

93 

Module  design 

70 

Coding 

85 

Module  test 

78 

Sub-system  integration 

70 

System  integration 

85 

System  test 

93 

Operation 

63 

Table  21 

Method  o£  Assigning 

Task 

Pet  Frequency  Reported 

Action  Most  of  the  Time  Seldom/Kever 

Task  assignment  given 
to  the  project  team 

In  writing 

77 

23 

Required  completion  dates 
were  included  with  each 
team  assignment 

92 

8 

Team  assignments  were  pre- 
pared in  such  a way  that 
the  relationship  of  a cask 
to  the  next  higher  level 
task  was  clearly  delineated 

62 

38 

Task  assignments  were  given 

Co  Che  individual  team  mem- 
bers In  writing 

54 

46 

Required  completion  dates  were 
Included  with  each  Individual 
team  member's  assignment 

79 

21 

Individual  team  member  written 
task  .-isslgnments  were  prepared 
In  such  a way  that  the  rela- 
tionship of  a task  to  the  next 
higher  level  task  was  clearly 
delineated. 

67 

33 

Time  and  taak  aaatgned  with  verification  37 
or  modification  of  time  allotted  being 
provided  by  Individual  team  mead>eca  aa 
a matter  of  procedure. 

Individual  team  member  provided  a time 
estimate  for  each  task  assigned  IS 

No  effort  was  made  to  determine  time  19 

requlrecMnts  for  Individual  tasks 


Formal  'Review.  In  only  7\  of  the  pro- 
jects reporting  were  no  formal  reviews 
held.  The  average  number  of  reviews  held 
per  project  was  between  3 and  4 with  many 
projects  holding  S.  There  were  no  sur- 
prises. As  a general  rule,  the  more  costV 
the  project,  the  more  reviews.  Table  23 
shows  the  types  of  reviews  and  the  per- 
centage of  projects  that  reported  using 
them.  An  additional  column  has  been  added 
to  show  the  percent  of  projects  that  used 
the  review  mechanism  in  establishing  a 
baseline. 


Table  23 

Formal  Revlews/Baaallnes 


Pet  Reported 

Reviews  Using 

Pet  Reported 
Establlahlng 
Baseline 

Systems  requirement  review 

56 

18 

Systems  design  review 

63 

9 

Preliminary  design  review 

85 

32 

Critical  design  review 

78 

32 

Formal  qualification  review 

30 

0 

Other  review! 

7 

0 

No  formal  revlewa  conducted/ 
no  baaellnes 

7 

27 

Table  24  attempts  to  answer  some  basic 
questions  about  the  review  proceeding,  the 
degree  of  formality  and  the  attendance. 


Informal  Reviews.  Reviews  between  the 
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Table  24 


Table  26 


Formal  Review  Proceedings 


Walk-Through  Frequency 


Pet  Frequency  Reported 

Action  Most  of  the  Time  Seldom/Never 


Type  of  Walk-Through 

Pet  Reported 

Formal  documenta- 
tion was  provided 

In  advance  of  each 
review 

88 

12 

Reviews  took  place 
on  schedule 

85 

15 

Top  management  attended 
formal  reviews 

42 

58 

Customer /user  attended 
formal  reviews 

79 

21 

Project  manager  attended 
formal  reviews 

93 

7 

An  Independent  review 
team  was  used 

23 

77 

manager  and  his  supervisor  were  also  a 
common  occurence.  Table  25  shows  that  961 
of  the  project  managers  engaged  in  this 
activity  with  varying  degrees  of  regularity. 


Table  25 

Frequency  of  Informal  Reviews 


Design  reviews 

46 

Coding  reviews 

38 

Did  not  use  walk-throughs 

38 

Table  27 

Walk-Through  Scheduling 


Frequency 

Pet  Reported 

Dally 

4 

Weekly 

4 

Monthly 

0 

As  required 

54 

Table  28 

Walk-Through  Attendance 


Frequency 

Pet  Reporting 

Dally 

7 

Weekly 

37 

Monthly 

22 

As  required 

27 

No  informal  reviews 

4 

Configuration  Management: 

Sixty-four 

percent  of  the  development  oroiects  re- 
portoil  usinc  some  form  of  software  con- 
Figiiration  management  system.  However, 
about  one- third  of  these  were  of  an  infor- 
mal nature  and  only  42t  resorted  to  a 
configuration  control  board.  Baseline 
data  was  included  in  Table  23. 

Walk-throughs.  This  technique  or  pro- 
cedure as  defined  by  Gerald  Weinberg 
(1971]  was  used  on  a number  of  the  pro- 
jects. Table  26  reflects  frequency. 

Table  27  scheduling,  and  Table  28  attend- 
ance. 


Attendance 

Pet 

Deslxn 

Type  Walk-Through 
Coding  Other 

Peer  programmers  or 
analysts 

45 

45 

18 

Programmer  or  analyst 
trainee 

14 

14 

5 

Programmer  or  analyst 
supervisors 

36 

32 

0 

Project  manager 

37 

18 

0 

Standard  monitors 

5 

9 

0 

Top  level  manager 

5 

0 

5 

User/customer 

9 

1 

0 

In  response  to  a question  asking  whether 
or  not  walk-through  minutes  were  main- 
tained it  was  found  that  records  were  kept 
in  the  design  walk-throughs  - 351  of  the 
time,  and  in  coding  walk-throughs  - 91  of 
the  time. 
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Directing  and  Motivating. 


Without  exception  the  project  manager  was 
responsible  for  the  technical  quality  of  the 
system  under  development.  Beyond  that  his 
responsibility  and  authority  wore  not  abso- 
lute. As  can  be  seen  in  Table  29,  the  proj- 
ect managers'  authority  was  most  circum- 
scribed when  it  came  to  hiring  and  firing, 
though  this  is  not  particularily  surprising. 
We  failed  to  determine  if  the  project  manag- 
er had  certain  prerogatives  in  selecting  (in 
or  out)  present  employees  of  the  firm,  but 
it  is  logical  to  assume  that  such  authority 
did  exist  to  some  extent. 


Table  29 

Project  Manager's  Responsibilities 


Project  Manager  Was  Responsible  For: 

Pet 

Technical  quality 

100 

Hire  and  fire  assigned  personnel  (within 
firm  policy) 

28 

Evaluate  performance  of  Individual 
personnel 

68 

Administration,  budget,  etc. 

80 

Allocating  computer  resources 

80 

Meeting  schedule  commitments 

96 

Negotiating  specification  changes  with 
customer 

92 

Making  a profit  (l.e.,  operating  within 
budget) 

48 

The  matrix  organization,  the  most  preva- 
lent form  in  the  projects  reporting,  undoubt- 
edly contributed  greatly  to  the  next  statis- 
tic (See  Table  30)  for  in  that  form,  the 
line  of  authority  generally  extends  direct- 
ly to  the  line  or  staff  manager  providing 
the  resource. 

Had  we  been  aware  of  the  extent  to  which 
matrix  organizational  forms  were  being  em- 
ployed we  would  have  made  some  attempt  to 
determine  not  only  which,  if  any, "labeled" 
management  or  motivational  techniques  were 
being  used,  but,  how  their  application  dif- 
fered when  employed  in  a matrix,  as  opposed 
to  a line  or  staff,  organization. 

At  any  rate,  the  results  of  the  survey 
(See  Table  31)  indicated  that  the  following 
techniques  or  procedures  were  being  employ- 
ed by  project  managers  in  the  percentages 
indicated. 

Approximately  lOt  of  the  projects  survey- 
ed reported  at  least  some  union  membership 
.nmong  the  project  personnel,  however,  union 
membership  had  no  recognizable  affect  on 
the  projects. 


Table  30 

Project  Manager's  Authority 
(A  Coaiposlte  Picture) 


Position  Description 


Pet  Assigned 


Full  time,  report  to  project  suinager 

Full  tlae,  report  outside  project 
manager's  organization 

Full  time,  outside  contractor/consultant 

Part  tlSM,  report  to  project  manager 

Part  time,  report  outside  project 
manager's  organization 

Part  tlae,  outside  contractor/consultant 


Table  31 

Management  Techniques 


Technique  Pet  Used 

Management  by  objectives  [Drucker,  1954]  62 
Management  by  exception  31 
Job  enrichment  [Herzberg,  1977]  7 
Incentive  and  suggestion  programs  14 
Open  door  policy  69 


Deliverables  and  Successes. 

The  measure  of  success  can  be  an  arbi- 
trary or  subjective  thing  and  so  it  was  with 
our  survey.  There  are  of  course  degrees  of 
success  and  one  man's  success  may  well  be 
another  man's  failure.  We  asked,  "Overall, 
how  well  do  you  think  that  this  project  met 
the  project  managers  major  goals:  to  de- 
liver on  time,  within  budget,  and  meeting 
the  requirement  of  the  system,  where  the 
final  software  product  is  reliable,  main- 
tainable, and  useable?" 

We  received  the  replies  listed  in  Table 


Though  every  project  was  not  yet  complete 
they  had  all  reached  that  state  of  "done- 
ness" that  the  managers  felt  confident 
enough  to  provide  an  on-time  or  months- 
late  input.  Just  under  SOt  of  the  projects 
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Table  32 
Project  Outcona 


Table  33 

Percentage  of  Production  Tine 


Goal  Attainment  Pet 


Extremely  well 

35 

Very  well 

26 

Good 

13 

Fair 

4 

Poor 

4 

Failed 

13 

had,  or  were  forecast  to  be,  completed  on 
time.  A similar  number  were  late  and  one 
project  was  delivered  early. 

The  average  number  of  months  late  was  8 
and  the  average  cost  overrun  was  251.  The 
major  causes  of  project  slippage  in  order  of 
importance  were  given  as;  changing  require- 
ments, unreasonable  or  poor  initial  esti- 
mates, and  limited  authority  over  resources. 
Cost  overruns  were  not  limited  to  late  pro- 
jects, however,  with  201  of  the  on  time  de- 
liveries also  exceeding  initial  estimates. 
The  project  that  came  in  early  also  came  in 
under  cost.  There  is  probably  a moral 
there. 

Since  no  measureable  standards  of  reli- 
ability and  maintainability  had  been  estab- 
lished these  requirements  were  met  when  the 
systems  produced  the  desired  outputs.  _Thrae 

projects  provided  a one  year  warranty  and 

another  indicated  an  implied  warranty  as 
long  as  the  firm  doing  the  development  work 
was  in  the  employ  of  the  user.  The  average 
production  rate  measured  in  lines  of  code 
per  programmer  per  day  was  15.  The  cost  per 
line  varied  from  $8.00  to  $200.00  - aver- 
aging $101.00.  This  number  should  not  be 
construed  as  contradicting  or  refining  the 
company  figure  given  on  a previous  page  nor 
does  it  convey  anything  more  significant. 

As  a final  question  we  asked:  What  per- 
cent of  production  time  was  spent  in  spec- 
ific areas  of  the  development  cycle?  The 
replies  are  summarized  in  Table  33. 

Opinions  on  Improving  Project  Management 

At  the  end  of  each  section  of  the  ques- 
tionnaire we  asked  the  participants  for  an 
opinion  on  what  could  or  should  be  done  to 
improve  the  state-of-the-art  in  software 
engineering  project  management.  Our  final 
section  contains  a selection  of  commen- 
taries. 

The  proper  preparation  of  requirement 
specifications  was  of  major  concern  to  most 
of  the  project  managers.  The  answers  given 
reflected  the  complaint  that  specifications 


Pec  par 

Area  Funeclon 


Requirement  epeciflcatlon 

11 

Preliminary  design 

12 

Detail  design 

21 

Progreamd.ng  (snd  unit  testing) 

24 

Integration 

19 

System  tasting 

13 

are  all  too  frequently  incomplete,  ambigu- 
ous, and  inconsistent  and  plagued  with  seem- 
ingly arbitrary  changes.  In  regard  to  this, 
one  large  manufacturer's  comment  was,  "Make 
sure  it  [the  requirement  specification]  is 
a joint  commitment  and  effort  by  customers 
and  developer".  Another  software  develop- 
ment manager  stated  "If  possible,  the  sys- 
tem definition  should  be  firmed  up  earlier 
and  held  so  that  changes  after  the  start  of 
software  development  are  only  minor  adjust- 
ments rather  than  redirection  of  the  ef- 
fort". No  requirement  specification  was 
written  in  a requirement  specification 
language.  However,  one  participant  be- 
lieves that  the  form  of  the  requirement 
specification  is  "...relatively  unimpor- 
tant, unless  the  customer  wants  to  impose  a 
design.  Otherwise,  any  readable  statement 
of  the  requirement  will  do,  as  long  as  it  is 
clear  and  complete". 

We  received  more  comments  on  planning 
than  on  any  other  management  function.  Many 
of  these  comments  were  similar  to  this  one: 
"Devote  more  effort  to  planning.  Involve 
programmers/analysts  to  a greater  degree. 
Require  full  documentation  of  plan.  Con- 
tinue to  review  plan  as  development  pro- 
ceeds". Another  project  managers  approach 
was  to:  "Emphasize  involvement  and  contri- 
bution of  the  "people  doing  the  work"  in 
the  planning  function". 

When  asked  what  changes  should  be  made 
in  the  way  technical  decisions  were  arrived 
at  concerning  programming  techniques,  a 
dichotomy  appeared  to  exist.  At  one 
extreme  a need  was  expressed  for  "more 
automated  tools;  more  formal  walk-throughs"; 
and  in  supporting  opinions  other  respon- 
dents suggested:  "review  what's  in  general 
use  and  publish  recommended  methods  as  a 
standard.  No  such  document  currently  ex- 
ists", and  "get  dollars,  survey  available 
methods,  apply  and  weed  out  losers".  How- 
ever one  project  manager  stated:  "I  would 
prevent  outside  'experts'  from  imposing 
their  pet  techniques  and  controls  in  areas 
where  they  are  Inappropriate,  misguided, 
and  costly". 
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The  matrix  form  of  organization  was 
clearly  considered  the  best  suited  to  soft- 
ware development  efforts.  However,  by  the 
opinion  that  the  form  of  the  organization 
was  not  as  important  as  a "single  clear 
leadership  role"  was  a sentiment  often  ex- 
pressed. Several  managers,  In  response  to 
a question  on  how  they  would  organize  for  a 
software  development  in  the  future  said 
"more  and  better  planning". 

Asked  what  action  they  would  take  if  it 
was  within  their  power  to  make  changes  or 
initiate  research  in  the  area  of  staffing, 
the  project  managers  indicated  they  would: 
'.'experiment  with  more  specialized  staff- 
ing. i.e.,  with  a number  of  functional 
roles  rather  than  a single  group  of  pro- 
grammers/analysts", "investigate  what 
personnel  backgrounds  lead  or  tend  to  lead 
to  high  performance  ADP  personnel"  and  im- 
plement studies  in  "how  to  maintain/im- 
prove communications". 

Most  of  the  project  managers  in  respond- 
ing on  the  subject  of  improved  project 
control  were  in  agreement  with  the  contrib- 
utor who  stated:  "establish  definite  pack- 
ages of  work  and  affect  regular  reviews  with 
milestones" . 

On  the  subject  of  improvements  in  the 
way  projects  were  directed,  comments  were 
very  sparce.  One  analyst  who  provided  us 
with  an  earlier  quote  firmly  believes  the 
necessity  for  "greater  delegation  of  auth- 
ority, responsibility",  and  "clearer  assign- 
ment of  responsibility". 

Finally,  the  project  managers  were  asked 
to  list  some  of  the  lessons  learned  from 
their  project.  The  responses  to  this  ques- 
tion were  voluminous  and  varied  however, 
there  was  one  response  that  clearly  stands 
out  as  a summary:  "establish  requirements 
clearly  before  designing;  establish  design 
clearly  before  programming".  "Give  more 
attention  to  planning  and  to  effort/cost 
estimates.  Need  greater  documentation  of 
decisions,  specifications,  [and]  design". 
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